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SUMMARY 

A wide variety of Space Station functions will be managed via computerized 
controls. Many of these functions are at the same time very complex and very 
critical to the operation of the Space Station. The Environmental 
Control and Life Support System is one group of very complex and critical 
subsystems which directly affects the ability of the crew to perform their 
mission. Failures of the Environmental Control and Life Support Subsystems 
are to be avoided and, in the event of a failure, repair must be effected as 
rapidly as possible. Due to the complex and diverse nature of the subsystems, 
it is not possible to train the Space Station crew to be experts in the 
operation of all of the subsystems. By applying the concepts of 
computer-based expert systems, it may be possible to provide the necessary 
expertise for those subsystems in dedicated controllers. In this way, an 
expert system could avoid failures and extend the operating time of the 
subsystems even in the event of failure of some components, and could reduce 
the time to repair by being able to pinpoint the cause of a failure when one 
cannot be avoided. 

A study was undertaken by Life Systems, Inc. to investigate the application of 
expert system concepts to fault diagnosis on Environmental Control and Life 
Support subsystems. An operating water recovery subsystem. Vapor Compression 
Distillation, was used as the specific example in the study. A detailed fault 
analysis was prepared and methods of applying expert system concepts were 
developed. An auxiliary computer was used in a demonstration of these 
concepts with an operating Vapor Compression Distillation Subsystem. Six 
examples were successfully executed with the subsystem, illustrating that all 
of the areas of fault diagnosis could be addressed with a computer-based 
expert system. The demonstration examples were recorded on video tape. 
Finally, an analysis of the potential application of these expert fault 
diagnostic concepts to other Environmental Control and Life Support Subsystems 
was performed. It showed that there is a great deal of similarity among the 
subsystems and, therefore, an anticipated high degree of applicability to all 
the subsystems in that group. 


INTRODUCTION 

The subsystems being developed for the Environmental Control and Life Support 
System (ECLSS) of the Space Station are designed to be low maintenance, long 
life, self-regulating units which would require little attention by the crew 
in normal operation. As a result, the crew would be ill-prepared to deal with 
extraordinary occurrences that could be potentially mission-threatening. 

The complexity of the subsystems makes fault diagnosis and repair a job for an 
expert. Rather than trying to train a crew to be experts on all such 
subsystems, artificial intelligence in the form of an Expert Fault Diagnostic 
(EFD) system would be a preferred alternative. Such a system could constantly 
monitor the processes and recognize faults, or potential fault conditions, and 
apply the knowledge of a process expert to avoid, mitigate or at least 
pinpoint the cause of such faults. In the area of ECLSS, extension of 
operating time in the face of component failures can be especially important. 


1 



£ife Systems. Jhc. 


Project Goals 

The program was set up to study the possible application of expert system 
concepts to fault diagnosis on a Vapor Compression Distillation Subsystem 
(VCDS). The subsystem was to be analyzed to incorporate the VCDS developer's 
fault diagnostic knowledge into a set of expert rules. Application of these 
rules could then improve the reliability, efficiency and maintainability of 
the VCDS. In addition, an analysis of the applicability of generic portions 
of the VCDS expert rules to another ECLSS was to be performed. 

Program Accomplishments 

A detailed analysis of the Vapor Compression Distillation (VCD) process (see 
Figure 1), its components and failure modes, resulted in a series of fault 
trees identifying more than 500 specific faults. From these fault trees, 
expert rules were developed specifying the logic used by an expert in 
avoiding, preventing, detecting, isolating, correcting and tolerating each 
type of fault. A demonstration of these expert rules was developed on an 
auxiliary computer interfacing in real time with an operating VCDS. Six 
examples were selected and successfully implemented to illustrate the 
application of the EFD concept at every level of fault management. A video 
tape of the examples was prepared. 

Other subsystems of the ECLSS were analyzed to identify components similar to 
those on the VCDS where the developed expert rules could also be applied. 

Such components were found in every ECLSS, showing that the EFD concept has 
wide applicability. Additional analyses were done in regard to the 
implementation of this concept on existing controllers on the ECLSS and within 
the hierarchy of Space Station controllers. The results of these analyses are 
included in this report. 


Final Report 

This report is organized into five major sections. The first presents 
background information on the VCD subsystem under study and expert systems and 
fault diagnostics in general. The second section covers the fault analysis 
prepared on the VCDS. The third section deals with issues that must be 
addressed when considering the application of expert systems to a process. 

The fourth section describes the demonstration experiment and the examples 
chosen to illustrate the EFD concepts, and the last section presents the 
results of the analyses concerning the extension of the concept to other 
subsystems of the ECLSS. 


BACKGROUND 

Objectives of this study were an analysis of a subsystem of the ECLSS group of 
the Space Station for application of expert systems' concepts to the area of 
fault diagnostics and the demonstration of those concepts with an operating 
subsystem. The subsystem used in this study was a VCDS, designated Vapor 
Compression Distillation, Enhanced Advanced Preprototype Configuration 
(VCD2B) , developed under previous portions of this contract. 
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Expert Systems 

The term "expert systems" refers to an area of artificial intelligence which 
attempts to capture the knowledge of an expert, organize that knowledge, and 
make it available to others. It is this concept that is to be applied to the 
VCDS and the problem of fault diagnosis. 

There are some general guidelines that can be used when trying to determine 
whether an expert system can be applied to a particular problem. If there are 
just a few individuals with special knowledge, if there is a significant and 
narrowly focused problem, and if the problem involves heuristics and judgment 
then expert systems can generally be applied to good advantage. In this case, 
the question is the application of expert systems to a VCDS and specifically 
to the area of fault diagnostics. Following the above guidelines, this would 
seem to be a good candidate for an expert system application. One final 
guideline in applying expert systems is whether the expert knowledge can be 
integrated into the target system. At first glance, this may seem to be a 
trivial or even nonsensical question, but in fact, it is very important. If 
the knowledge of the expert cannot be brought to bear on the problem in a 
practical way and in the environment in which it is required, then the 
resulting system will be useless. The demonstration experiment developed as 
part of this study answers this question for the VCDS. 

Many previous studies of expert systems as applied to an ECLSS have used 
process simulations as a test article in the demonstrations. While some of 
these have achieved a high degree of fidelity, they are nonetheless 
simulations of processes and not actual subsystems. In this study, an actual 
operating VCDS is used as a test vehicle for demonstrating expert systems 
concepts . 


Fault Diagnostics 

For any given subsystem, the area of fault diagnostics covers a wide variety 
of fault handling methods (see Table 1). Expert system concepts can be 
applied to some degree at each level. Table 2 lists the types of expert 
knowledge that would be applied using an expert system at each level of fault 
diagnostics. The demonstration developed in conjunction with this study 
illustrates each of these concepts using an operating ECLSS, specifically, the 
VCDS. 


Vapor Compression Distillation Subsystem 

The VCDS is a phase change water recovery process developed for the Water 
Management System (WMS) of the Space Station ECLSS (see Figure 2) . Its 
purpose is to recover potable water from onboard generated wastewater. It has 
been shown to be a highly efficient, low-specific energy consumption technique 
for accomplishing such water recovery. Figure 3 diagrams the VCD concept. 

The VCD process achieves its high efficiency by recovering the latent heat of 
vaporization from the evaporation-condensation cycle. It does this by 
compressing the vapor and condensing it on a surface in thermal contact with 
the evaporator. This process is carried out at a low temperature by 
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TABLE 1 FAULT DIAGNOSTIC LEVELS^ 


Operational 

Implementation 

Level Function 


Brief Description 


1 


2 

3 

4 

5 


6 


Fault Avoidance Design to avoid faults, e.g., 

- Prevent human errors 

- Eliminate weak links 

- Monitor interfaces 

Fault Prediction Predict fault will occur or is beginning 

Fault Detection Detect failure or symptom of failure 

Fault Isolation Identify what specifically failed 

Fault Correction Correct fault or provide detailed instructions 

(or Correction to enable correction 

Instructions) 

Fault Tolerance Tolerate faults without human intervention 


(a) First for specific operating modes (steady-state) and later during mode 
transitions. 
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TABLE 2 

Function 
Fault Avoidance 
Fault Prediction 
Fault Detection 
Fault Isolation 

Fault Correction 

Fault Tolerance 


Cal 

EXPERT SYSTEM CONCEPTS APPLIED TO FAULT DIAGNOSTICS v ' 


Brief Description 

Knowing how to prevent a fault from occurring (recurring) 

Knowing how to recognize that a fault is beginning 

Knowing how to recognize that a fault has occurred 

Knowing how to identify specifically the failed 
component 

Knowing how to correct a fault or provide instructions 
for correction 

Knowing when and how to ignore faults and continue safe 
operation 


(a) The demonstration illustrates each of these concepts using an operating 
ECLSS, specifically VCDS. 





• Equip. Atm. Cooling 

• Refrigerator/Chiller 

• Freeier 
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• Cold Water Outlet • Trash Compactor 
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• Airlock Depress./Repress. • Emergency Breathing Packs 
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FIGURE 3 VAPOR COMPRESSION DISTILLATION CONCEPT 
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maintaining the condenser and evaporator at a very low pressure. Figure 4 
shows the detailed mechanical schematic for the VCDS. The subsystem 
draws pretreated wastewater from the waste storage tank through the waste feed 
valve, labeled VI, and into the distillation unit where the evaporation- 
condensation process occurs. The fluids pump provides positive circulation 
for all fluids in the system. Excess wastewater is pumped from the evaporator 
section back to the waste storage tank for recycling or alternately through 
the recycle filter tank for removal of concentrated solids. The required low 
pressure in the distillation unit is maintained by periodic purging through 
the purge valve V3. Condensate is removed by the fluids pump and delivered to 
the post-treatment facility. If the product water quality is not adequate, 
then the output stream is redirected through valve V2 and reprocessed through 
the distillation unit. Due to the zero-gravity design requirement, the 
evaporator, condenser and condensate collector are rotated to provide the 
desired phase separation and liquid control. Figures 5 and 6 show the actual 
VCD mechanical hardware. 

VAPOR COMPRESSION DISTILLATION FAULT ANALYSIS 

The first step in implementing an expert system is to obtain the knowledge of 
the expert which relates to the problem to be addressed. The failures of the 
VCDS were analyzed in two ways: (1) by reviewing past VCD test programs and 

the failures which occurred during that testing, and (2) by doing a complete 
and detailed fault analysis of all of the components which make up the VCDS. 

Past Vapor Compression Distillation Faults 

Eight different sources of historical data on the VCDS were reviewed covering 
the period from April, 1983 through December, 1986. Three categories of VCDS 
failure were identified: a decrease in the water production rate, a 

less-than-acceptable water quality, and a decrease in the process efficiency. 
The process efficiency decrease could manifest itself either as an increase in 
the specific energy of the subsystem, an increase in the power consumption of 
the subsystem and/or a water production rate decrease. In all, 13 distinct 
faults were classified and these are listed in Table 3. For each of these 
faults the logic used in identifying and classifying it was specified and from 
that logic EFD rules for each fault diagnostic level were developed. Table 4 
lists an example of the fault diagnostic rules developed based on past faults. 
It is clear by the low number of past faults identified that an expert system 
could not be based on this information alone. As a result, a detailed 
analysis of the failure modes of the VCDS was undertaken. 

Vapor Compression Distillation General Fault Analysis 

In organizing the fault analysis, a series of generic categories were 
identified. Figure 7 illustrates the breakdown of all possible faults on the 
subsystem into these categories. As can be seen from Table 5, the past VCD 
faults that were identified can be related to the generic fault descriptions. 
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TABLE 3 SUMMARY OF PAST VCD SUBSYSTEM FAILURES 
FROM ALL PREVIOUS DEVELOPERS 

• C/M I Internal Power Supply 

• Q1 Sensor /Diaphragm Sticking 

• Motorized Valve Sticking 

• Fluids Pump Gearbox Fouling 

• Centrifuge Timing Belt Breakage 

• Centrifuge O-Ring Belt Breakage 

• Centrifuge O-Ring Belt Slippage 

• Centrifuge Bearing Fouling 

• Magnetic Drive De lamination and Decoupling 

• Fluids Pump Recycle Tubing Wear 

• Check Valve Sticking 

• Conductivity Sensor Signal Conditioning 

• C/M I Overheating and Software Failure 


(a) Eliminating those faults due to building power, TSA and operator error. 
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TABLE A EXAMPLE OF VCD EXPERT FAULT DIAGNOSTIC ROUTINE 
DESCRIPTIONS BASED ON PAST FAULTS 


Fault 


Diagnostic Level 


Description 


Fluids Pump Recycle Tubing Avoidance Replace pump tubing at regular 

Wear maintenance intervals 


Prediction Partial drydown cycles occurring 

at more frequent intervals 


Detection 


Isolation 


Correction 

Tolerance 


Repeated cycling to partial 
drydown 

Increasing fluids pump speed 
corrects the problem, decreasing 
it causes problem to return 

Replace pump tubing 

Cycle waste feed valve open and 
closed periodically (5 min/2 min) 
to limit intake of water while 
still pump out some fluid (inter- 
mittant partial drydown) or 
increase fluids pump speed 
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All Faults (Global) 


Actuators and 
• Other Components® 
Faults 


/ 


• Imbalance 

• Misalignment 

• Rubbing 

• Wear 

• Fouling (c) 

• Breakage® 1 * 


Incipient 

Faults® 3 * 


Degradation 

Faults 


Leakage 


c. 


• Fails Open 

• Fails Intermediate 
Position 

• Fails Last Position 

• Fails Closed 


Position 

Error 

Faults 


• Fails Energized 

• Fails De-Energized 


• Fails Zero 

• Fails Low Scale 

• Fails Full Scale 

• Fails Drifting 

• Fails Calibration Drifting 


\ 


Electrical 
Hardware and 
Sensor 
Faults 


(a) Mechanical components which are not sensors or actuators such as filters or storage tanks. 

(b) Faults which can be identified at their initial occurrence. 

(c) Includes solids precipitation sludge formation, blockage, plugging and sticking failures. 

(d) Subdivided into mechanical breakage and performance failure or “breakage’.’ 


FIGURE 7 GENERIC FAULT CLASSIFICATION 
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TABLE 5 GENERIC EQUIVALENT OF PAST VCDS FAULTS 


Past VCDS Fault 

Generic Fault Description 

Faults 
Peculiar 
to VCDS 

Faults 
Generic 
to ECLSS 

C/M I Computer Power Supply 

C/M I Failed De-Energized 

__ 

X 

Ql Sensor/Diaphragm Sticking 

Tank Degradation (Alignment) 

X 

- 

Motorized Valve Sticking 

Valve Degradation (Fouling) 

- 

X 

Fluids Pump Gearbox Fouling 

Gearbox Degradation (Fouling) 

- 

X 

Centrifuge Timing Belt Breakage 

Drive Motor Subassembly Breakage 

- 

X 

Centrifuge 0-Ring Breakage 

Drive Motor Subassembly Breakage 

- 

X 

Centrifuge 0-Ring Belt Slippage 

Drive Motor Subassembly Degradation (Rubbing) 

- 

X 

Centrifuge Bearing Fouling 

Centrifuge Degradation (Fouling) 

- 

X 

Magnetic Drive De lamination/ 

Drive Motor Subassembly Breakage 

- 

X 

Decoupling 




Fluids Pump Recycle Tubing Wear 

Pump Tubing Degradation (Wear) 

X 

- 

Check Valve Sticking 

Check Valve Degradation (Fouling) 

- 

X 

Conductivity Sensor S/C 

C/M I Failed Drifting 

- 

X 

C/M I Overheating and Software 

C/M I Failed De-Energized and C/M I Loss of 

- 

X 

Failure 

Programmed Control 

— 

— 

Totals 


2 
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The format used in performing the analysis was a series of fault trees. 

Figure 8 represents the fault tree for the VCDS using Orbital Replacement 
Units (ORUs) as the lowest level component. In order to utilize an expert 
system in the areas of fault tolerance and fault correction, it is necessary 
to isolate failures down to a component level rather than an ORU level. Only 
by identifying individual components would it be possible to determine the 
action to be taken in tolerating or correcting any failures. 

The VCDS was examined in detail, including both the pretreat and post-treat 
assemblies, and 109 individual components were identified. Failure modes of 
each component were identified and a total of 536 possible faults were listed 
for all of the components of the VCDS. Out of this number, 55 distinct types 
of faults were identified and, using the generic fault classifications, 30 
distinct generic types were listed. Due to the size of the analysis and 
quantity of data. Table 6 shows only a representative portion of a fault tree 
defined for the VCDS components. 

As had been done following the analysis of the past VCD faults, the logic used 
in analyzing each fault type was again specified. From this fault tree 
analysis and the expert logic, the generic fault diagnostic routines to be 
used by the expert system were once again identified. Tables 7, 8 and 9 
represent this progress from the fault tree to the expert fault diagnostic 
routines for one particular example failure. Similar descriptions were 
developed for each of the 30 classified generic types of faults. 

A thorough analysis of the possible subsystem faults is critical to a 
successful implementation of EFDs. Without such an analysis it is not 
possible to formulate the rules which are at the heart of such a system. The 
VCD general fault analysis, thus, stands as the basis for the expert system 
implementation and the rules which govern its operation. 

APPLICATION CONSIDERATIONS 

In implementing an expert system, it is necessary not only to examine the 
knowledge that an expert uses in approaching a problem, but also the methods 
employed in dealing with situations. A review of the methods that a subsystem 
developer might use in analyzing a failure can significantly influence the 
structure of the system rules and knowledge base in the implementation. 

Fault Analysis by a Subsystem Expert 

In general, a subsystem developer approaches the failure of the system in 
three phases. First is detection of the failure. Here, he might observe 
abnormal operation, in this case, decreases in water production rate or water 
quality or process efficiency. He might also be aware of warning or alarm 
messages issued by the subsystem controller, or sensor readings which deviate 
from the normal range. In isolating a fault, he might additionally look at 
actuator positions compared to their normal positions, as well as sensor 
readings compared to the associated actuator position. He may also perform a 
detailed analysis of each possibility and, if necessary, a component-by- 
component verification. Finally, to determine the ultimate cause of the 
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TABLE 6 DISTILLATION UNIT FAULT DEFINITION FOR 
PHASE CHANGE (VCD) SYSTEM 


Generic 

Component 

Component 

Qty. 

Fault Description 


Generic 

Specific 

Fault 

Qty. 

Distillation 

1 

Centrifuge Breakage 

Mechanical 

I 

Assembly 


Centrifuge Degradation 

• Imbalance 

1 




• Misalignment 

1 




• Rubbing 

1 




• Wear 

1 




• Evaporator Fouling 

1 



(a) 

Centrifuge Drive Breakage 

Mechanical 

I 



Centrifuge Drive Degradation 

• Imbalance 

1 




• Misalignment 

1 




• Rubbing 

1 




• Wear 

1 



Compressor Breakage 

Mechanical 

1 



Compressor Degradation 

• Imbalance 

1 




■ Misalignment 

1 




• Rubbing 

1 




• Wear 

1 




• Fouling 

1 



Outer Shell Leakage 

• Waste Water 

1 




• Product Water 

1 




• Vacuum 

1 

Drive Motor 

1 

Drive Motor Breakage 

Mechanical 

1 

Subassembly 


Drive Motor Degradation 

• Imbalance 

1 




• Misalignment 

1 




• Rubbing 

1 




• Wear 

1 



Drive Motor Electrical Failure 

• Fails De-Energized 

1 




• Fails Low Speed 

I 




• Fails Drifting Speed 

1 




• Fails High Speed 

1 



Magnetic Coupling Breakage 

• Mechanical ... 

1 




• Performance^ 

1 



Magnetic Coupling Degradation 

• Imbalance 

1 




• Misalignment 

1 




• Rubbing 

I 




continued- 



(a) Two-stage belt drive for speed reduction. 

(b) Decoupling. 
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Table 6 - continued 





Fault 

Description 


Generic 

Component 





Fault 

Component 

Qty. 


Generic 


Specific 

qty. 

Temperature 

2 

Sensor 

Breakage 

Mechanical 

2 

Sensor 


Sensor 

Degradation 

Fouling 

2 



Sensor 

Leakage 

Vacuum 

2 



Sensor 

Electrical Failure 


Fails De-Energized 

2 





• 

Fails Zero 

2 





• 

Fails Low Scale 

2 





• 

Fails High Scale 

2 





• 

Fails Drifting 

2 





• 

Fails Calibration 
Drifting 

2 

Speed Sensor 

2 

Sensor 

Breakage 

Mechanical 

2 



Sensor 

Degradation 

Fouling 

2 



Sensor 

Leakage 

Vacuum 

2 



Sensor 

Electrical Failure 

• 

Fails De-Energized 

2 





• 

Fails Zero 

2 





• 

Fails Low Scale 

2 





• 

Fails High Scale 

2 





• 

Fails Drifting 

2 





• 

Fails Calibration 
Drifting 

2 

Liquid Level 

1 

Sensor 

Breakage 

Mechanical 

1 

Sensor 


Sensor 

Degradation 

Fouling 

1 



Sensor 

Leakage 

Vacuum 

1 



Sensor 

Electrical Failure 

• 

Fails De-Energized 

1 





• 

Fails Zero 

1 





• 

Fails Low Scale 

1 





• 

Fails High Scale 

1 





• 

Fails Drifting 

1 






Fails Calibration 
Drifting 

1 

Motor Current 

1 

Sensor 

Breakage 

Mechanical 

1 

Sensor 


Sensor 

Electrical Failure 

• 

Fails De-Energized 

1 





• 

Fails Zero 

1 





• 

Fails Low Scale 

1 





• 

Fails High Scale 

1 





• 

Fails Drifting 

1 





• 

Fails Calibration 
Drifting 

1 


Total Generic 8 

Total Specific 

86 

Faults Possible 

Faults Possible 
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TABLE 7 EXAMPLE OF FAULT TREE ANALYSIS 


Generic 

Component 

Fault Description 

Fault 

Component 

QtV- 

Generic 

Specific 

qty. 

Liquid Filter 

1 

Filter Degradation 

Fouling 

1 



Filter Leakage 

Product Water 

1 
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TABLE 8 EXAMPLE OF EXPERT LOGIC 

Fault Type Logic Description 

Filter Degradation Flow rate decreases over time 

Upstream pressure increases over time 

Time since last maintenance approaches or exceeds 
scheduled maintenance interval 

Physical inspection shows fouling 

Replacement returns operation to normal 
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TABLE 9 EXAMPLE OF GENERIC EXPERT FAULT DIAGNOSTIC ROUTINES 


Fault Diagnostic Level 

Filter Degradation Avoidance 

Prediction 


Detection 

Isolation 

Correction 

Tolerance 



Replace filter at regular maintenance 
intervals 


Flow rate has decreased significantly 
from original value 
Pressure has increased significantly 
from original value 

Time since last maintenance is approach- 
ing the maintenance inverval 

Flow rate is low 

Bypassing filter increases flow rate 
Replacing filter increases flow rate 
Inspection of filter shows fouling 

Replace filter 

(None) 
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failure, the developer might be required to disassemble and inspect the 
subsystem components suspected of causing the failure, or replace a component 
and retest the subsystem in order to verify the actual cause of failure. 

Figure 9 represents graphically the order in which a subsystem developer might 
approach the analysis of a fault. 

An expert system would have to be able to mimic the expert, in both his data 
gathering and his logical reasoning, to perform similar functions. Direct 
observation of the subsystem operation can be performed using electromechanical 
sensors to take the place of the human senses used by the expert. The senses 
of vision, hearing, touch, taste and smell can be replaced by corresponding 
sensors which might record the temperature, pressure, flow, vibration, noise 
and other comparable quantities. It should be noted, however, that totally 
replicating a human expert's senses would require a substantial increase in 
the number of subsystem sensors required. Table 10 illustrates how a great 
number of additional sensors, above those normally on the subsystem, would be 
required on the VCD in order to replicate what a human expert might observe on 
the system. In most cases, all of these sensors would not be required. 

Readings from some of the sensors might be able to be deduced by readings 
taken elsewhere on the system. Also, when reviewing the actual expert rules 
implemented in a particular subsystem, many of these sensors would not be 
required to support those rules. The sensor complement of any subsystem being 
considered for expert applications must be reviewed for the addition of 
sensors required by the expert system rules. Generally, additional sensors 
will be required above those normally used for fault detection and safety 
shutdown alone. 

A subsystem expert might also make use of high level or composite views of the 
subsystem in monitoring its performance. An expert system could also implement 
the same function by generating composite sensors. These sensors would be 
combinations of the normal sensor complement on the subsystem which would give 
a single value representative of the quality of operation of a given aspect of 
the subsystem. Tables 11 and 12 give examples of composite sensors that might 
be implemented on the VCDS and those faults which could be identified using 
those sensors. 

The logical reasoning of an expert that is used in analyzing a fault can also 
be duplicated in an expert system. Logic programming can reason from rules 
listed in a rule base similar to the heuristic rules that a human expert might 
use when approaching a problem. If the rules of experience are not successful 
in pinpointing a problem, an expert generally resorts to reasoning from the 
theory of operation. This could also be implemented in the rule base of an 
expert system. While it may not be possible for an expert system to actually 
do the detailed tear down and inspection or substitution and verification, it 
would be possible for an expert system to recommend probable causes and 
suggest a course of action for repair. In this way, an expert system could 
duplicate virtually all of the functions of an expert in analyzing a subsystem 
failure . 

Finally, an expert system should be able to easily add to its knowledge base 
when new rules and new failure modes are identified. Rules could be added to 
the knowledge base not only to assist in identifying future failures more 
easily, but also to improve performance of the system during normal operation. 
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FIGURE 9 DEVELOPER'S LOGIC IN ANALYZING A FAULT TYPE 
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TABLE 10 ADDITIONAL SENSORS REQUIRED ON VCDS TO PROVIDE 
INFORMATION EQUIVALENT TO HUMAN EXPERT 


VCD Subsystem ORUs 


Sensor Type 

Pret. 
Tank A 

Pret. 
Tank B 

FCA/ 

Pret. 

Urine 
Stor . 

Wash. 

Stor. 

Fluids 

Pump 

Distl. 

Unit 

FCA / PCA/ Recycle FCA/Post- 
VCD VCD Tank Treat 

Post-Treat 

Subassy. 

BAMU 

Total 

Temperature 

- 

- 

2 

- 

- 

1 

1 

- 

- 

- 

- 

- 

- 

4 

Flow 

1 

1 

2 

1 

1 

3 

- 

3 

- 

1 

3 

2 

1 

19 

Vibration 

- 

- 

2 

- 

- 

1 

1 

- 

- 

- 

1 

- 

- 

5 

Noise 

- 

- 

2 

- 

- 

1 

1 

- 

- 

- 

1 

- 

- 

5 

Concentration 

1 

1 

- 

- 

- 

- 

- 

- 

- 

1 

- 

- 

- 

3 

Current 

- 

- 

2 

- 

- 

1 

1 

- 

- 

- 

1 

- 

- 

5 

Voltage 

- 

- 

2 

- 

- 

1 

1 

- 

- 

- 

1 

- 

- 

5 

Liquid Level 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

13 

Pressure 

___ 

_1 

___ 

— w 

__ . 

_1 

__ __ 








_2 


3 

4 

13 

2 

2 

10 

6 

4 

1 

3 

8 

3 

2 
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TABLE 11 POTENTIAL VCDS COMPOSITE SENSOR LIST 


Composite Sensor Name 

Composite 

Sensor 

Symbol 

Participating 

Sensors 

Participatir 

Sensors 

Symbols 

Condenser saturation condition 

KS1 

Condenser pressure 

PI 



Condenser temperature 

T1 

Evaporator pressure 

KS2 

Compressor delta P 

P2 



Condenser pressure 

PI 

Gross liquid level 

KS3 

Liquid level 

LI 



Still drive current 

11 

Evaporator saturation condition 

KSA 

Condenser pressure 

PI 



Compressor delta P 

P2 



Evaporator temperature 

T2 • 

Distillation unit drive condition 

KS5 

Centrifuge speed 

S3 



Still drive speed 

SI 



Still drive current 

11 



Condenser temperature 

T1 



Liquid level 

LI 



Time in normal mode 

Z5 

Fluids pump drive condition 

KS6 

Pump speed 

S2 



Pump drive current 

12 



Time on. flu ids pump 

Zll 



Liquid level 

LI 



Waste storage quantity 

Ql. Q2 

Recycle loop solids level 

KS7 

Waste storage quantity 

Ql. Q2 



Time in normal mode 

Z5 



Total time on recycle tank 

Z12 



Condenser temperature 

T1 

Centrifuge delta T 

KS8 

Condenser temperature 

T1 



Evaporator temperature 

T2 
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TABLE 12 VCDS FAULTS ISOLATED BY COMPOSITE SENSORS 


Composite Sensor Name 

Composite 

Sensor 

Symbol 

Condenser saturation condition 

KS1 

Evaporator pressure 

KS2 

Gross liquid level 

KS3 


Evaporator saturation condition KS4 


oo Distillation unit drive condition KS5 


Fluids pump drive condition 


KS6 


VCDS Fault (s) Isolated 


• High condenser pressure due to faulty purge valve V3 

• Out-of-tolerance water production rate 

• Used in computing evaporator saturation condition KS4 

• Out-of-tolerance water production rate 

• Waste feed valve (VI or V6) failure to close during 
partial drydown 

• Loss of C/M I control 


• High evaporator pressure due to distillation unit vacuum 
leak 

• Out-of-tolerance water production rate 


Drive subassembly breakage 
Drive subassembly degradation 
Centrifuge breakage 
Centrifuge degradation 
Compressor breakage 
Compressor degradation 
Casing process fluid leakage 
Casing vacuum leakage . 

Fluids pump breakage'' 


Fluids pump degradation 


(b) 


• Fluids pump breakage 

• Fluids pump degradation 

• Pump casing vacuum leakage 

• Pump drive subassembly breakage 

• Pump drive subassembly electrical 


continued- 


(a) Loss of fluids pump drive such that recycle loop pumping capability is lost. 

(b) Pump recycle loop tubing wear. 
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Table 12 - continued 


Composite Sensor Name 

Composite 

Sensor 

Symbol 

Fluids pump drive condition 

KS6 

- continued 


Recycle loop solids level 

KS7 

Centrifuge Delta T 

KS8 


VCDS Fault (s) Isolated 


• Pump tubing breakage 

• Pump tubing degradation 

• Impending high solids fouling of VCDS 

• Recycle/filter tank changeout 

• Loss of waste feed fluid . . 

0 Waste storage tank degradation^ 

• Tank quantity sensor failure 


ro 

VO 


(a) Such as rolling diaphragm sticking. 
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Expert System Automation 

In applying an expert system for fault diagnostics on any particular 
subsystem, it is important that an analysis be done specifically for the 
expert system application. The normal analyses performed in designing and 
building a subsystem generally do not address those issues that are directly 
related to the EFD system. A typical Failure Mode and Effects Analysis (FMEA) 
may identify certain failure modes of the subsystem and their effects, but the 
form of the data is such that it would make expert system construction very 
difficult. The fault analysis used in this study (i.e., fault trees) lends 
itself far more readily to translation into the expert system rules and the 
reasoning logic. 

The complement of sensors on the subsystem is also a critical item and this 
needs to be addressed likewise, specifically for the expert systems 
implementation. It is recommended that a second analysis of subsystem sensors 
be performed after the rules are formulated for the expert system so that the 
desired degree of expert implementation, as well as the types of failures to 
be detected, can influence the types and quantities of sensors required. As 
mentioned earlier, it is often possible to derive the value of certain 
subsystem parameters in a number of ways using different types of sensors. 

Once the expert systems analysis has been completed, the selection of the 
actual subsystem sensor list can take into account both the requirements of 
the operating control and fault detection as well as the expert systems 
requirements. 

The response time of an expert system when used in a fault diagnostic 
application is especially critical. In order to safely avert a shutdown of 
the subsystem, the expert system must be able to analyze a situation, identify 
the fault, isolate the cause, and effect corrective action within the time 
frame normally allowed before a safety shutdown of the subsystem. In most 
cases this is only a matter of a few seconds. Systems which consist of a 
large number of rules will need very fast processors to maintain an adequate 
response time. Similarly, slower processors would limit the amount of expert 
knowledge which could be applied to a subsystem. The demonstration system was 
found to respond adequately in spite of the additional communications delay 
resulting from the implementation being in an auxiliary processor, separate 
from the subsystem controller, however, the rule base for this system was 
quite restricted. 

The rule base for an expert system can use up a great deal of the available 
storage in a typical subsystem controller. When adding expert knowledge at 
that level, this situation must be considered. Either additional storage must 
be made available, or the rule base will have to be restricted to some degree. 

The subsystem rules that are identified and are to be implemented for a 
particular subsystem can be implemented a number of different ways. The 
initial formulation of the rules should be independent of the implementation 
and should not directly rely on a particular coding scheme or a particular 
language of implementation. The demonstration system developed under this 
study takes advantage of that situation by using an implementation that 
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closely relates to the current architecture of the subsystem controller. The 
rules that were formulated could just as well have been implemented using a 
higher-level, more traditional artificial intelligence language. 

EXPERT DEMONSTRATION SYSTEM 

Once the fault analysis of the VCDS was completed and the rules governing the 
implementation of that knowledge were formulated, it was then possible to 
select examples from that rule base which could be demonstrated using an 
operating VCDS. Six such examples were selected from among the EFD rules 
which were developed. These six examples illustrate the application of expert 
system concepts to all the operational levels of fault diagnostics. 

Overview 

An auxiliary computer was selected to implement the expert demonstration 
system. This was done to simplify the development work of the expert system 
by isolating any interaction between it and the normal control logic in the 
subsystem controller to a simple communications interface. Another advantage 
of this organization was a much clearer demonstration due to the additional 
color graphic displays which were available on the auxiliary computer. It was 
also possible to more easily separate the expert system logic from the normal 
subsystem controls and, thereby, contrast the difference in operation of the 
subsystem with and without the expert logic being activated. Also, there was 
an existing interface for communications between the controller and an 
auxiliary computer. No modification to the subsystem control software was 
necessary in order to add the expert demonstration system to the VCDS. No 
changes to the control algorithms were made, nor was any allowance made 
concerning response time of the expert system. In all cases, the subsystem 
controller reacted exactly as it has been configured, except for those 
instances when the expert system issued commands to override sensors or 
actuators. Any and all such commands used by the expert system were already 
available and understood by the subsystem controller. 

Figure 10 illustrates the overall organization of the demonstration system. 

The mechanical hardware of the VCDS interacts with the subsystem 
controller through the sensors and actuators. A RS-232C communications 
channel links the auxiliary computer with the subsystem controller. Through 
this communications link, the auxiliary computer can update itself with the 
current status of the subsystem sensors and actuators, as well as issue 
commands to the subsystem controller to take action when the expert system 
requires it. An additional significant note is the use of Intel 8086 family 
processors in both the subsystem controller and the expert demonstration 
system. The software for this expert system was also coded in the same 
programming language. Programming Language /Microcomputers (PL/M), that is used 
in the subsystem controller. The selection of PL/M as the implementation 
language allowed the use of an existing library of low level interface 
routines for both the Control/Monitor Instrumentation (C/M I) communications 
and the color graphic displays. In addition, the compiler for PL/M is highly 
optimized and generates efficient native code for the Intel processors 
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yielding a high execution speed which can be critical in an expert system 
implementation. The fact that the same family of microprocessors is used in 
both the auxiliary computer and the subsystem controller means that the expert 
system logic developed in this separate machine could be transported fairly 
easily and incorporated into the control software of the existing subsystem 
controller. The demonstration, therefore, not only would illustrate how 
expert system concepts can be applied to an existing subsystem, but also that 
they could be applied using existing subsystem controllers. 

Selected Examples 

Figure 11 shows a sample display screen developed for the expert demonstration 
system. This screen is the initial startup screen where one of the six 
examples can be selected for operation. All of the screens used in the expert 
demonstration system have the same general format. The bar at the top of the 
screen provides the current date and time, as well as the current status, of 
the subsystem, either normal, warning or alarm. This bar will actually change 
colors depending upon the status of the subsystem, being green for normal, 
yellow for warning or red for alarm. The area of the screen labeled 
"comments" is used for a narrative on the progress of the selected example. 

Any messages appearing in this area are generated by the supervisory routines 
and not by the actual expert system rules. The section of the screen labeled 
"advisory" contains those messages which are actually generated by the expert 
system rules themselves. It is this portion of the screen which will provide 
the operator with the fault correction, detection, isolation and other 
messages issued by the expert system. The final portion of the screen is used 
in various ways depending on the situation at hand. In some cases, it will 
contain a graphic representation of the subsystem itself where faulty 
components are highlighted for identification to the operator. In other 
cases, it may be used to show the trend of data over a period of time to 
explain to the operator why a certain action is required or why the expert 
system is taking a particular course of action at that time. 

Table 13 lists the six examples which were selected for the demonstration. It 
also indicates the areas of fault diagnostics which are illustrated using each 
particular example. As can be seen from the Table, the fault detection and 
isolation logic are required by the expert system for virtually every 
operation. Before fault correction action can be taken or before a decision 
can be made on tolerating a fault, the expert system must first determine the 
actual component which has failed. Thus, some of the examples illustrate a 
number of levels of fault diagnostic application. The six examples were 
chosen so that all six levels of fault diagnostics can be illustrated in this 
demonstration. 

Figures 12 through 17 and Tables 14 through 19 provide the details of each 
individual example. In each instance, each situation was demonstrated with 
the expert system disabled and then again with the expert system enabled. The 
resulting effects on the subsystem in each case are recorded for each example. 
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FIGURE 11 EXPERT DEMONSTRATION SAMPLE SCREEN - STARTUP SCREEN 
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TABLE 13 EXAMPLES OF EXPERT SYSTEM CONCEPTS 







Fault 




No. 

Description 

Avoidance 

Prediction 

Detection 

Isolation 

Correction 

Tolerance 

Action 

1 

Speed Sensor Failure 



X 

X 


X 

Substitute derived value for failed 
sensor to avoid shutdown 

2 

Still Drive Motor Failure 



X 

X 



Identify failure and suggest 
corrective action 

3 

Waste Feed Valve Fouling 

X 


X 

X 

X 


Correct fault condition and perform 
periodic preventive maintenance 

4 

VPI Failure 



X 

X 


X 

Identify failure and substitute 
derived value for failed sensor 

5 

C/M I Component Failure 


X 





Predict impending fault allowing 
time to correct failure condition 

6 

Temperature Sensor Failure 



X 

X 


X 

Identify failure and permit 
continued operation for a limited 
period of time based on historical 
trend 
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TABLE 1A EXAMPLE 1 - SPEED SENSOR FAILURE 

Description: 

Speed Sensor, SI, is disconnected to simulate a failure of the sensor. 

. Demonstrates: 

How an expert can recognize the failure and continue operation by 
deriving a value for SI from sensor S3 reading and known interrelation of 
S1/S3. 

Without Expert: 

SI low alarm indication. 

Subsystem shuts down immediately. 

With Expert: 

Speed sensor is identified as failed. Subsystem continues normal 
operation. 

Value for SI is substituted by calculation using S3 reading. 

Control loop for still motor speed continues to function with no specific 
knowledge of a fault. 
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FIGURE 13 EXPERT DEMONSTRATION SAMPLE SCREEN - EXAMPLE 2 
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TABLE 15 EXAMPLE 2 - STILL DRIVE MOTOR FAILURE 


Description: 

Drive Motor is disconnected to simulate the failure of the motor. 
Demonstrates: 

How expert knowledge of the subsystem components can be used to more 
effectively identify and isolate the source of the failure, reducing the 
time to repair. Also shows how expert suggestions and recommendations 
can be incorporated into fault diagnostic logic. 

Without Expert: 

Subsystem shuts down immediately with only a single alarm message 
indicating Low SI reading. 

With Expert: 

Low SI reading is verified by comparable Low S3 reading, eliminating 
speed sensor failure as cause, identifying drive motor as failed 
component . 

Suggestions for verification of type of motor failure are presented, 
based on current being drawn by motor. 
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FIGURE 14 EXPERT DEMONSTRATION SAMPLE SCREEN - EXAMPLE 3 
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TABLE 16 EXAMPLE 3 - WASTE FEED VALUE FOULING 


Description: 

Waste Feed valve is physically inhibited from reaching a new position. 
Demonstrates: 

How expert knowledge of the subsystem operation can be used to correct 
fault conditions and prevent shutdown. Further, such knowledge can be 
used to perform preventative maintenance on some components. 

Without Expert: 

Failure to reach position causes an immediate shutdown of subsystem with 
an alarm message indicating the failure. 

With Expert: 

Valve is cycled back and forth three times to attempt to free up the 
valve. When this succeeds, demonstrated by removing the impediment, a 
cycle of automatic preventative maintenance is instituted. The subsystem 
continues normal operation. 
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TABLE 17 EXAMPLE 4 - VPI FAILURE 


Description: 

The failure of a valve position indicator is demonstrated by overriding 
the VPI sensor reading. 

Demonstrates : 

How multiple levels of expert knowledge can be applied to a failure. 

Also demonstrates how knowledge of the relations between sensors and 
actuators can be used to indirectly deduce the actual state of certain 
actuators (valves) . 

Without Expert: 

Failure to reach position is detected (erroneously) and an immediate 
shutdown of the subsystem occurs. 

With Expert: 

A fouled valve is first suspected and that corrective logic is attempted 
first. When this fails, the operation of the VPI is verified using 
knowledge of flow at FI. The failed VPI is identified and a value for it 
is substituted based on the measured flow. A warning message is issued 
identifying the VPI failure, but the subsystem continues to function. 


43 


ORIGINAL PAGL IS 
01 EGOR QUALITY 



FIGURE 16 EXPERT DEMONSTRATION SAMPLE SCREEN - EXAMPLE 5 
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TABLE 18 EXAMPLE 5 - C/M I COMPONENT FAILURE 


Description; 

Simulation of failure of C/M I cooling. 

High temperature would eventually cause a failure in the integrated 
circuits and the subsystem would shut down before this point is reached. 

Demonstrates : 

How an expert can recognize an impending failure and provide warning that 
a condition is developing which, if not corrected, could result in shut 
down of the subsystem. Also shows how expert knowledge can suggest 
possible causes to assist in correcting the problem. 

Without Expert; 

Temperature rises with no advance warning until fault detection limits 
are encountered. Subsystem eventually shuts down. 

With Expert: 

Initial warning is given based on trend of temperature before fault 
detection limits are reached. This provides additional time to correct 
the problem. Possible causes are listed to aid in locating and 
correcting the problem. 
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TABLE 19 EXAMPLE 6 - TEMPERATURE SENSOR FAILURE 


Description: 

Failure of a temperature sensor is simulated by overriding the sensor 
reading. 

Demonstrates: 

How expert knowledge of the subsystem can identify invalid readings and 
can make a judgment concerning continuing operation. 

Without Expert: 

Subsystem shuts down immediately. 

With Expert: 

Instantaneous change in temperature reading is recognized as invalid and 
a result of sensor failure. No backup sensor is available, but based on 
theoretical maximum rate of change of temperature, subsystem is permitted 
to continue operation for a period of two hours, thus extending operating 
time and allowing more time for maintenance preparations. 
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The first example (Figure 12, Table 14) illustrates the failure of a sensor by 
manually disconnecting the speed sensor SI. This example is used to 
demonstrate the fault tolerance of a subsystem by recognizing the failure of 
the sensor and deriving a value from a known relationship with another sensor 
on the subsystem. In this case, operation of the subsystem can be continued 
without a shutdown. The expert system automatically substitutes a calculated 
value for the failed sensor and normal control algorithms within the subsystem 
controller continue without any interruption. Note also that an automatic 
maintenance scheduling algorithm can easily be implemented using an expert 
system. The fault detection and isolation mechanisms inherent in the EFD 
function makes such maintenance scheduling easy to implement. Note also that 
on the graphic display associated with this example the operator is presented 
with a picture of the subsystem where the failed component is highlighted so 
it may be easily located and repaired or replaced. 

The second example (Figure 13, Table 15), that of still drive motor failure, 
is related to the first in that it illustrates how an expert system must be 
able to distinguish between a number of faults which may have similar 
symptoms. In this case, the failure of the still drive motor also results in 
a low reading on the speed sensor. In this case, however, the comparison 
between the two speed sensors can isolate for the expert system the fact that 
it is not a sensor failure in this case, but a failure of the drive motor. 

Note also that the advisories issued by the expert system provide the operator 
with additional information in determining the exact cause of the failure. In 
this case, the expert system itself cannot determine the absolute cause of the 
failure because it does not have access to a drive current sensor reading. 

Even in cases such as this, however, the expert system can provide additional 
diagnostic assistance by providing a check list to the operator which could be 
used to determine the ultimate cause of the subsystem fault. 

The third example (Figure 14, Table 16) concerns a degradation in the 
operation of a valve on the subsystem. The condition simulated here was one 
that actually occurred during testing and was identified in the analysis of 
past VCD failures. Under certain conditions, deposits can build up on the 
valves which inhibit their operation. When this occurs, the valve may not 
reach the position that is requested of it by the subsystem controller. 
Ordinarily, this would cause an immediate shutdown of the subsystem. When 
this problem is encountered by an expert, it is generally alleviated by 
cycling the valve back and forth a number of times to free up the sticky 
mechanism. The expert system rules implemented in this case attempt to 
perform the same operation to free up the valve. This example also 
illustrates how an expert system can be set up to provide automatic preventive 
maintenance on various components. In this case, if the corrective action on 
the sticking valve is successful, the expert system then periodically cycles 
the valve back and forth to prevent such a buildup from occurring again. 

The fourth example (Figure 15, Table 17) also relates to the valve operation 
and shows how various expert system rules can interact. In this example, the 
failure of the Valve Position Indicator (VPI) at first looks to the expert 
system as the same situation as the sticky valve in the previous example. As 
a result, the expert system first attempts the same corrective action, but 
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when this fails, then looks further on into the system to try to determine 
whether it is a VPI that has failed. In this case, it uses the knowledge of 
flow in a particular line which is downstream of the valve to determine the 
actual position of the valve. Using this expert knowledge of the subsystem, 
it can then continue operation of the subsystem and avoid a shutdown in the 
event of this sensor failure. 

The fifth example (Figure 16, Table 18) illustrates how an expert system can 
be used to predict a fault before it actually occurs. In this case, the 
internal temperature of the subsystem controller is being monitored. An 
excessively high temperature could result in misoperation of the subsystem 
controller and loss of control of the subsystem. To avoid this, alarm limits 
are placed and automatic shutdown is initiated if this temperature goes too 
high. A failure of the cooling system is simulated in this instance causing a 
drastic increase in the temperature over time. By projecting the temperature 
at a future point in time, the expert system is able to give additional 
warning to the operator to allow him to correct the problem before shutdown of 
the subsystem is necessary. The expert system is also able to advise the 
operator as to the probable cause of this failure of the cooling system. 

Based on its knowledge of the subsystem, the historical trend of the 
temperature, and the maintenance history of the cooling system in particular, 
the expert system is able to provide a prioritized list of possible causes of 
the failure. The expert system is not able to automatically correct this 
problem, but it is able to guide the operator. 

The last example (Figure 17, Table 19) also illustrates a problem with the 
temperature on the subsystem. The difference in this case is that the 
temperature changes dramatically in a very short period of time rather than as 
a longer-term trend. The expert system can, thus, recognize a failure of the 
temperature sensor due to the fact that it has knowledge of the maximum rate 
of change of this particular variable on the subsystem. Also illustrated in 
this example, is the concept of a safe operating time. While the expert 
system does not immediately shut down the subsystem, it also cannot permit the 
subsystem to continue operation indefinitely. The sensor which has failed, 
while not critical to the short-term operation of the system, is critical in a 
longer-term sense. Therefore, the expert system permits operation in this 
degraded mode for only a limited amount of time. 

All of these examples illustrate how the concept of EFDs and an expert system 
implementation can extend the operating time of the subsystem or reduce the 
down time of a subsystem in the event of a critical failure. It can do this 
by making use of the knowledge of a subsystem expert and by communicating that 
knowledge to the system operator. 


Conclusions 

The expert demonstration system implemented under this study has shown that 
EFD concepts can extend the operating time of an ECLSS. It can also reduce 
the time to repair the subsystem by identifying the source of the failure to 
an operator. It has also shown that these concepts can be applied to existing 
subsystems by using an unmodified VCDS and subsystem controller to implement 
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the expert system rules selected in this demonstration. Also, by using an 
architecture similar to that which exists on the current generation of 
subsystem controllers, it has demonstrated that the expert systems and EFD 
concepts illustrated here can be implemented using existing controllers. 

APPLICATION TO OTHER ENVIRONMENTAL CONTROL AND LIFE SUPPORT SUBSYSTEMS 

The demonstration with an operating VCDS has shown the practicality and 
effectiveness of an EFD system applied to one particular subsystem of the 
ECLSS. An analysis of the architecture and components of the other ECLSS 
was undertaken to assess the applicability of these concepts to other 
subsystems. 


Implementation Within Existing Controllers 

Figure 18 illustrates the software architecture of a typical subsystem 
controller of the ECLSS group. The operating system software has 
responsibility for all common functions to be performed, such as reading 
values of sensors and outputting commands to actuators. These operations can 
be tailored to individual subsystems by tables describing, for example, the 
sensor or actuator complement in the datta base for each subsystem. The 
application software contains the custom processing required for a particular 
subsystem, such as control loop and device driver handling or special 
calculations. Table 20 shows some detail of this organization for a typical 
subsystem. This modular software architecture lends itself quite well to the 
addition of expert logic. Table 21 shows where expert systems could be 
implemented within such a controller. Note that the fault isolation, fault 
correction, fault prediction and fault tolerance modules have been reassigned 
from the operating system fault diagnostic group to the application side of 
the organization. This reflects the detailed knowledge of the subsystem 
expert that is used in developing the expert system rules. To properly 
perform these functions, detailed and specific subsystem knowledge is 
essential. 

Thus, the software architecture of the ECLSS controllers can be seen to be 
compatible with the implementation of expert systems. Further, since the 
described organization is common to a number of ECLSS controllers, the 
application of expert fault logic to other ECLSSs should be a reasonable 
expectation. 

Generic Faults Within Environmental Control and Life Support Subsystems 

An investigation of all of the subsystems of the ECLSS group was conducted to 
identify those components which could be considered common, or generic, among 
all of the subsystems. Figure 19 shows the functional partitioning used in 
the analysis, and Table 22 lists the distribution of the generic faults 
previously identified from the VCDS study. It can readily be seen that this 
analysis also supports the idea of applying similar expert systems 
logic to all of the other groups of the ECLSS. 
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TABLE 20 EXPERT SYSTEM IMPLEMENTATION WITHIN EXISTING CONTROLLERS 


Current Organization 


Operating System 

Fault Diagnostic Group - Fault Detection 

Fault Isolation . . 
Fault Correction, 3 . 
Fault Prediction, ? 
Fault Tolerance 

Application 

Process Control Group - Pressure Control 

Recycle Control 
Quantity Control 
Quality Control 
Level Control 
Percent Solids Control 

Special Sensor - Solids Concentration 

Calculation Group Quantity Processed 


(a) Not currently implemented. 

(b) Triple redundant sensors only. 
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TABLE 21 EXPERT SYSTEM IMPLEMENTATION WITHIN EXISTING CONTROLLERS 


Future Organization 


Operating System 

Fault Diagnostic Group - Fault Detection 


Application 


Fault Diagnostic Group - Fault 

Fault 

Fault 

Fault 


Isolation 

Correction 

Prediction 

Tolerance 


Expert Systems Implementation 


Process Control Group - Pressure Control 

Recycle Control 
Quantity Control 
Quality Control 
Level Control 
Percent Solids 
Control 

Enhanced Control Expert Systems Implementation 


Special Sensor 
Calculation Group 


- Solids Concentration 
Quantity Processed 
Composite Sensors j 
Sensor Substitution [ 


Expert Systems Implementation 
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TABLE 22 


DISTRIBUTION OF GENERIC. FAULTS WITHIN 
SPACE STATION ECLSS U) 


EVA Safe 

Generic Fault Description ARS APCCS THCS WMS WRS Support Haven 


Valve Motor Degradation X 

Valve Motor Breakage X 

VPI Failure X 

Check Valve Degradation 
Valve Degradation 

Valve Breakage X 

Valve Position Error X 

MCV Body Leakage — 

MCV Resin Depleted 


MCV Resin Blockage (Degradation) 

MCV Body Breakage 

Sensing Element Leakage 

Sensing Element Electrical X 

Sensing Element Degradation X 

Sensing Element Breakage X 

Pump Motor Electrical 

Pump Motor Degradation 

Pump Motor Breakage 

Tank Leakage 

Tank Degradation 

Tank Breakage 

Pump Degradation - 

Pump Breakage 

Pump Casing Vacuum Leakage 
Gearbox (Drive) Degradation 

Gearbox (Drive) Breakage 

Compressor (Pump) Breakage 
Compressor (Pump) Degradation X 

Filter Blockage (Degradation) X 

Filter Leakage X 


X X X X 

X X X X 

X X X X 

X X 

X X 

X X X X 

X X X X 

X 
X 

- - - X 

X 

X - X X 

X X X X 

X X X X 

X X X X 

- - X X 

X X 

X X 

X 
X 
X 

X X 

X X 

X X 

X X 

X X 

X 

X - - X 

X X X X 

X X X X 


X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 


X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 


(a) 01/09/86 Version. 

Abbreviations: 

APCCS = Atmosphere Pressure and Composition Control System 
ARS = Air Revitalization System 
EVA = Extravehicular Activity 
MCV = Microbial Check Valve 

THCS = Temperature and Humidity Control System 
WMS = Waste Management System 
WRS = Water Reclamation System 
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Table 23 describes the logic used in analyzing these generic faults. For this 
logic to be applied to a number of other subsystems, there must be a way to 
relate the general statements to the specific sensors and actuators used in 
each individual subsystem. In the context of expert systems, this means that 
the knowledge base developed must include specific relations between, and 
among, the individual sensors and actuators of each subsystem. The rule base 
would be specified in such a way as to describe the general relationships 
which are exploited in analyzing a particular fault. For example, in 
describing the logic involved in monitoring a filter, the rule base would 
contain statements such as: 

Rule 1. If downstream flow rate for filter is less than 50% of original 
flow rate, then filter is degraded. 

Rule 2. If upstream pressure for filter is greater than 150% of original 
pressure, then filter is degraded. 

Rule 3. If time since last maintenance for filter is greater than 80% 
of maintenance interval, then filter is degraded. 

The knowledge base would also contain specific facts concerning each subsystem 
such as : 

Fact 1. VCD-FLTR is a filter 

Fact 2. Downstream flow rate for VCD-FLTR is VCD-F1 

Fact 3. Upstream pressure for VCD-FLTR is VCD-P3 

Fact 4. Original flow rate for VCD-F1 is VCD-CALC2 

Fact 5. Original pressure for VCD-P3 is VCD-CALC5 

Fact 6. Time since last maintenance for VCD-FLTR is timer VCD-Z152 

Fact 7. THCS-FLTR is a filter 

Fact 8. Upstream pressure for THCS-FLTR is THCS-P6 
Fact 9. Original pressure for THCS-FLTR is THCS-CALC8 

Facts 1 to 6 relate specifically to the VCD filter, while facts 7 to 9 
identify a filter in the THCS. The inference engine would then use the rule 
for finding a degraded (blocked) filter on either subsystem using the specific 
facts identifying individual sensors where required. In some cases, certain 
facts may not be available. For example, the THCS in this example does not 
have a downstream flow sensor. The expert system would have to deal with 
these variances. It would also be advisable to include in the system some 
certainty, or confidence, factors associated with each fact or rule. Rule 3, 
for example, is not necessarily strictly true, but represents a "good guess." 
Missing information, such as the THCS flow sensor, could also be handled with 
such a factor. 

There are some situations which may have to be treated in a more specialized 
manner. For example, sensor breakage or degradation may not be able to be 
generalized for all sensors on all subsystems. However, it may be possible to 
treat all flow sensors, or all pressure sensors, the same. Further, there 
might be a class of rules applicable to all sensors, and then sets of rules 
which would apply only to certain types of sensors, such as flow or pressure 
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TABLE 23 SPECIFIC LOGIC USED IN ANALYZING A FAULT TYPE 


Fault Type 
Tank Breakage 


Tank Leakage 


Tank 

Degradation 


Sensor 

Breakage 


Sensor 

Degradation 


Sensor 

Electrical 


Sensor 

Leakage 


Motor 

Breakage 


Motor 

Degradation 


Logic Description 

Tank quantity does not change properly in relation to process 
operation. 

Loss of fluid outside of process is detected. 

Physical inspection shows serious mechanical failure of tank. 

Tank quantity does not change properly in relation to process 
operation . 

Loss of fluid outside of process is detected. 

Physical inspection shows no serious mechanical failure of 
tank. 

Tank quantity changes erratically or not at all in relation to 
process operation. 

No loss of fluid outside of process is detected. 

Physical inspection of bellows or rolling diaphragm shows 
fouling (sticking) . 

Sensor reading is not changing with process or is reading 
highly suspect value. 

Physical inspection shows mechanical damage. 

Sensor reading in relation to process varies in one direction 
over period of time. 

Replacement returns operation to normal. 

Sensor reading is not changing with process or is reading 
erratic values. 

Sensor calibration values are extreme or erratic. 

Sensor reading in relation to process varies over time. 
Replacement returns operation to normal. 

Loss of fluid outside of process is detected. 

Physical inspection shows failure of seal on sensor mounting. 
Replacement or re-seating returns operation to normal. 

Motor speed is erratic or zero. 

Motor sound changes characteristics. 

Motor vibration increases when running. 

Physical inspection shows mechanical damage. 

Motor speed becomes erratic. 

Motor sound changes characteristics. 

Motor vibration increases when running. 

Physical inspection shows no mechanical damage. 

Replacement returns operation to normal. 


continued- 
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Table 23 - continued 


Fault Type 

Motor 

Electrical 


Drive 

Breakage 


Drive 

Degradation 


Pump 

Breakage 


Pump 

Degradation 


Pump Leakage 


Valve 

Breakage 


Valve 

Degradation 


Valve 

Leakage 


Valve 

Position 

Error 


Logic Description 

Unable to start or stop motor. 

Unable to adjust speed of motor. 

Motor speed varies over period of time. 

Driven component's speed is erratic or zero. 

Driving motor is operating properly. 

Physical inspection shows mechanical damage. 

Driven component's speed is erratic. 

Driven component's sound changes characteristics. 

Driven component's vibration increases when running. 

Physical inspection shows no mechanical damage. 

Replacement returns operation to normal. 

Measured output is erratic or zero. 

Pump sound changes characteristics. 

Pump vibration increases when running. 

Physical . inspection shows mechanical damage. 

Measured output decreases or becomes erratic. 

Pump sound changes characteristics. 

Pump vibration increases when running. 

Physical inspection shows no mechanical damage. 

Replacement returns operation to normal. 

Loss of fluid outside of process is detected. 

Physical inspection shows failure of seal. 

Replacement or re-seating returns operation to normal. 

Loss of fluid outside of process is detected. 

Unable to change valve position. 

Physical inspection shows mechanical damage. 

Flow rate decreases over time for a given open position or 
flow rate is not zero with valve closed. 

Valve does not respond to initial attempt to change position, 
but does respond after repeated attempts. 

Physical inspection shows no mechanical damage. 

Replacement returns operation to normal. 

Loss of fluid outside of process is detected. 

Physical inspection shows failure of seal. 

Replacement or re-seating returns operation to normal. 

Valve does not respond to attempt to change position. 

Physical inspection shows no mechanical damage. 

Replacement returns operation to normal. 


continued- 
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Table 23 - continued 


Fault Type 


Logic Description 


Valve 

Position 

Indicator 

Electrical 


Check Valve 
Breakage 

Check Valve 
Leakage 


Check Valve 
Degradation 


MCV Breakage 


MCV Leakage 


MCV Resin 
Breakthrough 


Indicated valve position does not reflect last requested 
position. 

Process measurements or observation confirm actual position 
of valve is requested position. 

Replacement returns operation to normal. 

Loss of fluid outside of process is detected. 

Physical inspection shows mechanical damage. 

Loss of fluid outside of process is detected. 

Physical inspection shows failure of seal. 

Replacement or re-seating returns operation to normal. 

Forward flow rate decreases over time. 

Backward flow is non-zero when flow should be inhibited. 
Replacement returns operation to normal. 

Loss of fluid outside of process is detected. 

Physical inspection shows mechanical damage. 

Loss of fluid outside of process is detected. 

Physical inspection shows failure of seal. 

Replacement or re-seating returns operation to normal. 

Contamination of process is detected. 

Physical inspection shows breakthrough. 

Replacement returns operation to normal. 


MCV Resin 
Degradation 


MCV Resin 
Depletion 


Forward flow rate decreases over time. 

Contamination of process increases over time. 

Backward flow is non-zero when flow should be inhibited. 
Replacement returns operation to normal. 

Contamination of process is detected. 

Physical inspection shows no breakthrough. 

Replacement returns operation to normal. 


Filter 

Degradation 


Filter 

Leakage 


Flow rate decreases over time. 

Upstream pressure increases over time. 

Time since last maintenance approaches maintenance interval. 
Physical inspection shows fouling. 

Replacement returns operation to normal. 

Loss of fluid outside of process is detected. 

Physical inspection shows failure of seal. 

Replacement or re-seating returns operation to normal. 
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sensors. Substitution for failed sensors might be handled generally in some 
cases, or individually by special cases where general rules cannot be applied 
either due to a lack of the necessary sensors for substitution or the use of 
specialized sensors. 

Table 24 lists the identified generic faults and the areas within the Space 
Station ECLSS where the associated fault diagnostic routines could be applied. 

In general, it appears that the EFDs developed here will be easily applicable 
to a number of other ECLSSs. Both the general approach, as well as some of 
the specific rules formulated could be transferred to new applications. 

Location of Expert Logic 

This analysis has identified six specific types of expert systems routines 
that may be located within the Space Station ECLSS. They are fault avoidance, 
fault prediction, fault detection, fault isolation, fault correction, and 
fault tolerance. Fault avoidance attempts to prevent faults from occurring. 
This can be accomplished by monitoring the maintenance intervals required on 
the subsystems' components and ensuring that proper maintenance is performed, 
and also by preventing erroneous actions either by a human operator or by 
subsystem interaction. Fault prediction routines monitor the subsystems in an 
attempt to predict when a fault condition is beginning or when it might occur 
in the future. Fault detection routines monitor the subsystem for an imminent 
failure or symptoms of a failure as it is occurring. Fault isolation routines 
then attempt to determine specifically which component caused the failure. 
Fault correction routines attempt to maintain system operation by correcting 
the cause of the error. This may be accomplished by substituting for failed 
units or by entering an alternate operating mode which then corrects the 
condition of the subsystem which was detected as a fault. Fault tolerance is 
related to fault correction but it is not an attempt to actually correct the 
fault. Fault tolerance routines instead would attempt to continue operation 
of the subsystem, perhaps in a degraded mode, by bypassing the faulty 
components in some manner. 

Figure 20 illustrates the hierarchy of controllers which is the current Space 
Station automation concept. Within this hierarchy there are a number of 
locations where expert system routines might be applied. At the lowest level 
is the subsystem controller which provides localized control over individual 
subsystems such as oxygen generation or wastewater recovery. At a higher 
level is the system controller which monitors an entire family of subsystems. 
This controller provides a centralized, supervisory form of control. Above 
the system controller is the module controller which has responsibility for an 
entire module of the Space Station. This controller again would operate in a 
supervisory control mode providing high level commands to the individual 
system controllers and, from there, to the subsystems. Some types of expert 
system routines could be located in any one of the levels of control within 
the Space Station. 
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TABLE 24 


Fault Type 

Tank Breakage 
Tank Leakage 
Tank Degradation 

Sensor Breakage 
Sensor Degradation 
Sensor Electrical 

Sensor Leakage 


Motor Breakage 
Motor Degradation 


Motor Electrical 


Drive Breakage 
Drive Degradation 


Pump Breakage 
Pump Degradation 


Pump Leakage 


Valve Breakage 
Valve Position Error 
Valve Position 

Indicator Electrical 

Valve Leakage 


Valve Degradation 


Check Valve Breakage 
Check Valve Leakage 


APPLICATION OF GENERIC FAULT ROUTINES WITHIN 
SPACE STATION ECLSS 


Application 

All subsystems; any storage tank or temporary fluid 
reservoir having a level sensing element. 


All subsystems; any sensing elements. 


All subsystems which have fluid flow; any sensing elements 
related to fluid flow. 

All subsystems; any electrical motor controlled by the 
subsystem having a speed sensor and/or sound/vibration 
sensors . 

All subsystems; any electrical motor controlled by the 
subsystem having a speed or on/off sensor. 

All subsystems containing components which are driven 
through a power transfer system (gears, belts, pulleys, 
etc.); any power transfer assembly with associated 
speed, motor condition and/or sound /vibration sensors. 

All subsystems which have fluid pumps or compressors; any 
pump or compressor which has output sensors, and/or input 
and sound/vibration sensors. 

All subsystems which have fluid pumps or compressors; any 
pump or compressor which has an external fluid loss 
sensor. 

All subsystems; any valve, controlled by a subsystem, 
which has position sensors or fluid flow sensors. 


All subsystems; any valve which has an external fluid 
loss sensor. 

All subsystems containing corrosive or contaminated 
fluids; any valve, controlled by a subsystem, which has 
position sensors or fluid flow sensors. 

Any check valve which has associated flow rate sensors 
and/or external fluid loss sensor. 


continued- 
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Table 24 - continued 

Fault Type 

Check Valve Degradation 

MCV Breakage 
MCV Leakage 

MCV Resin Degradation 

MCV Resin Breakthrough 
MCV Resin Depletion 

Filter Degradation 
Filter Leakage 


Application 

Any check valve which has associated flow rate sensors. 
Any MCV having an external fluid loss sensor. 

Any MCV having associated contamination sensors and/or 
flow rate sensors. 

Any MCV having associated contamination sensors. 


All subsystems; any filter having an associated pressure 
and/or flow rate sensor. 

All subsystems; any filter having an external fluid loss 
sensor. 
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Location of expert system routines within the individual subsystems has some 
advantages. Localized expert routines could be made specific to the 
individual subsystem, rather than being stated generally. These routines can 
then employ a detailed knowledge of the subsystem operation. Without the 
delays inherent in the communications from the higher level controllers, these 
localized routines can also afford faster response to situations than those 
located in the higher level controllers. Localized routines would also be 
easier to test and verify, since there would be no interactions or conflicts 
among the routines in different subsystems. 

Centralized locations for expert system routines also afford some advantages. 
Located in the higher level controller these routines could provide 
supervisory functions that would not be possible if located in the subsystem 
controller. With a global view of the situations, the higher level 
controllers could apply the expert logic across subsystem boundaries. In some 
cases this may be the only way to properly identify faults which occur near 
the interfaces between subsystems. Fault avoidance routines include 
maintenance scheduling, external command verification, and management of 
interactions between subsystems. These functions would most appropriately be 
handled by a central controller. Since none of these functions require a high 
speed response, the additional communications delay from a higher level 
controller to the subsystem would not be a factor. Fault prediction routines, 
which tend to apply long term trend analysis in forecasting possible failures, 
also do not require short response time and, therefore, can also be centrally 
located. 

For other situations the choice of local versus central location is not so 
clear cut. Fault detection routines generally require a high speed response 
to prevent possible subsystem damage. For this reason, fault detection 
routines should in general be located locally within the subsystem controller. 
For detection of faults which may occur at subsystem boundaries or between 
subsystems, a central location for some of these fault detection routines 
would be necessary. Likewise, some fault isolation routines would need to be 
centrally located as well. Fault Isolation routines in general do not have a 
high speed response requirement, and all of these could be located in a 
central controller. However, fault correction and fault tolerance routines 
may require high speed response to prevent a subsystem shutdown and they, in 
turn, may depend on rapid isolation of a fault. Therefore, some of the fault 
isolation routines may need to be located locally within the subsystem 
controller. Fault correction and fault tolerance routines which prevent 
subsystem shutdown require high speed response and need to be located within 
the subsystem controller. Those fault correction and fault tolerance routines 
which operate at the interfaces between subsystems need to be located in the 
higher level controller. Such routines might make use of sensors in one 
subsystem to substitute for failed sensors in an adjacent subsystem. 

It appears, therefore, that a combination of locations for the expert system 
routines, both localized and centralized, provides for the best application of 
the expert systems concept within the Space Station ECLSS. 
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CONCLUSIONS 

A thorough study of fault diagnostics on a VCDS has shown how the concepts of 
EFD systems can extend the operating time of an ECLSS. An actual demonstra- 
tion using an operating VCDS confirmed this and also illustrated how aspects 
of such an expert system could also reduce the time required to repair a 
subsystem following a failure. Furthermore, this demonstration showed that 
existing subsystems could be enhanced with EFD logic and that this logic could 
be implemented in the current generation of subsystem controllers. 

An analysis of other ECLSSs has shown both similar controller architecture and 
similar complements of components leading to the conclusion that all ECLSSs 
are viable candidates for the addition of EFD logic. A study of the Space 
Station controller hierarchy with a view to the implementation of expert 
knowledge systems concluded that these concepts can and should be implemented 
both centrally, in higher level controllers, and locally, in the subsystem 
controllers, to most effectively apply the expert knowledge. 

Costs and Benefits 

Implementing an expert system can result in significant additional costs when 
developing a subsystem. Additional time will be required for analysis, 
additional sensors may need to be added to the subsystem, and additional 
testing will be required due to the more complex implementation and 
integration. The benefits of having such a system, however, can be 
substantial and dramatic. Improved process efficiency, through enhanced 
control and fewer subsystem shutdowns, as well as shorter downtime when 
repairs are required, can more than make up for additional costs of 
development . 

Lessons Learned 

During the course of this study a number of important lessons were learned. 
Anyone attempting to implement an EFD system should be aware of the following 
items. 

• The fault tree and definition of expert routines are the top level 
"assembly drawings" of the EFD blueprint package. 

• Application of expert systems will tend to increase the number of 
subsystem sensors desired. 

• An analysis of the subsystem must be performed specifically for EFD 
applications. The required information and insight does not "fall 
out" of other subsystem analyses (e.g., FMEA) . 

• Response time is of critical importance for fault correction and fault 
tolerance applications intended to prevent shutdown of a subsystem. 
This may affect the location and degree of EFDs when implemented. 
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• Knowledge acquisition is the most important and time consuming aspect 
of expert system applications. 

• Basing expert systems only on past experience may result in a system 
with very limited capability. 

• While fault isolation need only be performed to the ORU level for 
maintenance and repair, advanced EFD implementations require isolation 
to component level. 


Recommendations 

In order to further promote the addition of EFD logic in ECLSSs, it is 
recommended that a set of guidelines be developed for performing an analysis 
of a subsystem and applying the expert knowledge which is collected. Other 
subsystems of the ECLSS should be analyzed in detail for EFD and knowledge 
bases developed for each. Presently, subsystem developers are on their own in 
determining in what areas and to what extent expert knowledge of a subsystem 
is incorporated. Management of failures is generally limited to fault 
detection and safe shutdown of the subsystem. 

One possible way to ease the addition of expert knowledge might be to develop 
a traditional expert system (inference engine and rule base) that would be 
compatible with existing controller architecture and that could be easily 
included in the controllers. Such a system might be developed in a 
transportable software language such as "C" or "Ada." Availability of such a 
package would reduce the implementation and testing costs associated with the 
addition of expert knowledge, since the rules themselves would be the only 
component needing verification. 

In any event, it has been shown that EFD systems are beneficial to the 
long-term reliability of the Space Station systems. The form of implementa- 
tion is not important. It is not necessary to wait for traditional artificial 
intelligence machines or languages to be trimmed down to subsystem controller 
size. Nor, is it necessary to add expert knowledge only on large machines and 
at a high level. Some restrictions on the degree of expertise included may 
result from limitations of equipment or expenditure, but these concepts, at 
least to some degree, can and should be added to the next, if not the current, 
generation of subsystem controllers. 
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